Everything about Numerically Stable totally explained
In the
mathematical subfield of
numerical analysis,
numerical stability is a desirable property of numerical
algorithms. The precise definition of
stability depends on the context, but it's related to the accuracy of the algorithm.
Sometimes a single calculation can be achieved in several ways, all of which are algebraically equivalent in terms of ideal real or complex numbers, but in practice when performed on digital computers yield different results. Some calculations might damp out approximation errors that occur; others might magnify such errors. Calculations that don't magnify approximation errors are called
numerically stable. One of the common tasks of numerical analysis is to try to select algorithms which are
robust — that's to say, have good numerical stability.
Example
As an example of an unstable algorithm, consider the task of adding an array of 100 numbers. To simplify things, assume our computer only has two digits of precision (for example, you can only represent numbers in the hundreds as 100, 110, 120, etc.).
The obvious way to do this would be the following pseudo-code:
sum = 0
for i = 1 to 100 do
sum = sum + a[i]
end
That looks reasonable, but assume the first element in the array is 1.0 and the other 99 elements are 0.01. In pure math, the answer would be 1.99. However, on our two-digit computer, once the 1.0 was added into the sum variable, adding in 0.01 would have no effect on the sum, and so the final answer would be 1.0 – not a very good approximation of the real answer.
A stable algorithm would first sort the array by the absolute values of the elements in ascending order. This ensures that the numbers closest to zero will be taken into consideration first. Once that change is made, all of the 0.01 elements will be added, giving 0.99, and then the 1.0 element will be added, yielding a rounded result of 2.0 – a much better approximation of the real result.
Forward, backward, and mixed stability
There are different ways to formalize the concept of stability. The following definitions of forward, backward, and mixed stability are often used in
numerical linear algebra.
Consider the problem to be solved by the numerical algorithm as a
function f mapping the data
x to the solution
y. The result of the algorithm, say
y*, will usually deviate from the "true" solution
y. The main causes of error are
round-off error,
truncation error and
data error. The
forward error of the algorithm is the difference between the result and the solution; in this case, Δ
y =
y* −
y. The
backward error is the smallest Δ
x such that
f(
x + Δ
x) =
y*; in other words, the backward error tells us what problem the algorithm actually solved. The forward and backward error are related by the
condition number: the forward error is at most as big in magnitude as the condition number multiplied by the magnitude of the backward error.
In many cases, it's more natural to consider the
relative error »
instead of the absolute error Δ
x.
The algorithm is said to be
backward stable if the backward error is small for all inputs
x. Of course, "small" is a relative term and its definition will depend on the context. Often, we want the error to be of the same order as, or perhaps only a few
orders of magnitude bigger than, the
unit round-off.
The usual definition of numerical stability uses a more general concept, called
mixed stability, which combines the forward error and the backward error. An algorithm is stable in this sense if it solves a nearby problem approximately, for example, if there exists a Δ
x such that both Δ
x is small and
f(
x + Δ
x) −
y* is small. Hence, a backward stable algorithm is always stable.
An algorithm is
forward stable if its forward error divided by the condition number of the problem is small. This means that an algorithm is forward stable if it has a forward error of magnitude similar to some backward stable algorithm.
Stability in numerical differential equations
The above definitions are particularly relevant in situations where truncation errors are not important. In other contexts, for instance when solving
differential equations, a different definition of numerical stability is used.
In
numerical ordinary differential equations, various concepts of numerical stability exist, for instance
A-stability. They are related to some concept of stability in the
dynamical systems sense, often
Lyapunov stability. It is important to use a stable method when solving a
stiff equation.
Yet another definition is used in
numerical partial differential equations. An algorithm for solving an evolutionary
partial differential equation is stable if the numerical solution at a fixed time remains bounded as the step size goes to zero. The
Lax equivalence theorem states that an algorithm converges if it's consistent and stable (in this sense). Stability is sometimes achieved by including
numerical diffusion. Numerical diffusion is a mathematical term which ensures that roundoff and other errors in the calculation get spread out and don't add up to cause the calculation to "blow up".
Further Information
Get more info on 'Numerically Stable'.
|
External Link Exchanges
Do you know how hard it is to get a link from a large encyclopaedia? Well we're different and will prove it. To get a link from us just add the following HTML to your site on a relevant page:
<a href="http://numerical_stability.totallyexplained.com">Numerical stability Totally Explained</a>
Then simply click through this link from your web page. Our crawlers will verify your link, extract the title of your web page and instantly add a link back to it. If you like you can remove the words Totally Explained and embed the link in article text.
As long as your link remains in place, we'll keep our link to you right here. Please play fair - our crawlers are watching. Your site must be closely related to this one's topic. Any kind of spamming, dubious practises or removing the link will result in your link from us being dropped and, potentially, your whole site being banned. |
